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I. SUMMARY 

- The addition of ISAM file capability to UTS will require 
12 man-months of effort, total, for design, implementation 
and checkout. 

The work will take six months. 

UTS is organized to permit clean addition of the ISAM 
file access method. Each access method is implemented 
by a self-contained content manager. Several access 
methods have been added recently at a cost of a few man- 
months effort each. 

UTS ISAM will be equivalent to XOS ISAM in performance. 
UTS ISAM will provide all functional capabilities available 
in XOS ISAM. 

UTS ISAM will provide automatic increase in file size (extends), 
as required, a feature not available in XOS ISAM. 
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II. INTRODUCTION 

The ISAM facility in UTS is a functional adjunct to the conventional 
multi-level keyed file capability whfch provides superior performance 
and requires less direct access storage for a special class of file mani- 
pulation. Current multi -level keyed files support all of the functional 
capabilities of ISAM (except for incidental differences involving the 
location of the key field and maximum key length). Enhanced perfor- 
mance is provided for sequential access of ISAM files by combining 
the "master-index" (of conventional keyed files) with the data. Thus, 
ISAM combines the space and sequential access speed advantages of 
consecutive files with the random access by keys of keyed files . 

A new file organization, I for ISAM, is provided as a new modular 
(isolatable) facility in UTS. This new file organization may be accessed 
either sequentially or randomly (by key). ISAM files must be created 
sequentially (sorted-key order), but may be updated randomly (by key). 

The KEYED files do not constrain keys to be within the data record and 
(currently) constrain key length to 31 byies. The ISAM files require keys to 
be contained within the data record, but allow keys to be up to 255 bytes 
in length. 

III. FUNCTIONAL DESCRIPTION OF ISAM FOR UTS 



The ISAM file organization may be accessed via the standard file CALs 
M:READ, M:WRITE and M:DELREC. Extensions are provided in three dis- 
tinct system areas: 

A modular file access routine for ISAM supplements the current 
set (individual modules currently manage consecutive, keyed, and 
random file organizations). 

The user control language is incidentally extended in batch (CCI) 
and on-line (TEL) to reflect the new organization (indexed) and the 

key length and key oosition within the data record. 
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Sysfem procedures (for Meta Symbol) are expanded to include 
the new organization and the key length and key position 
within the data record. 

Changes to the system procedures, CCI and TEL are only cosmetic additions 
to existing parameterized facilities and hence are essentially trivial. The 
only significant development is concentrated in the provision of a new, modular 
access routine. 

The control language extensions to CCI (Batch) and TEL (on-line) are as 

follows: 

TEI 
^ !SET deb params;INDEXED;BLKL=lll;KEYP=mmm;KEYL=nnn 

CCI 

^ lASSIGN deb params;INDEXED,(BLKLjII),(KEYP,mmm),(KEYL,nnn) 

INDEXED is a new organization and supplements the current set which are 

CONSECutive, KEYED and RANDOM. 

The value associated with KEYP indicates the key position relative to byte 
of the data record, while the value associated with KEYL indicates the 
key length from 1 to 255 bytes. The value of BLKL indicates block length* 
in bytes and will be rounded up to an integral number of 2048 byte pages for 
system use. 

The system procedures (reflected in the Metasymbol SYSTEM BPM) are extended 
as follows: 

M:DCB dcb,(INDEXED),(KEYP,mmm),(KEYL,nnn),(BLKL,lll) 

M:OPEN dcb,(INDEXED),(KEYP,mmm),(KEYL,nnn),(BLKL,Ill) 
No cosmetic changes are required for the procedures: 

MrREAD (read record) 

M:V/RITE (write record) 

M:DELREC (delete record) 



* The block length is assumed to be 2048 bytes (the current s>'stem standard) if 
unspecified. This value allows user blocking factor control up to 8192 bytes; 
logical records, hov/ever, can.be as large as user memory as contrasted to the 
XOS limit of 32, 767 bytes. 
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The KEYL (key length) value of INDEXED would be treafed exactly 
as KEYM (key max. length) is now for KEYED files. 

The index will automatically "prefer" RAD and the data will automatically 
"prefer" DISC, however, the space for each is allocated dynamically and 
no user file space specification is required for public files. No special 
utility is required for "reorganization" to eliminate overflow blocks; a 
standard PCL (peripheral control language) COPY will move an INDEXED 
file to any device, including OVER itself, to effect reorganization. Also, 
a standard LIMIT card specification for FPOOL (file pool) and IPOOL 
(index pool) will provide (and allow) space to hold high-level index blocks 
in core and minimize redundant (wasteful) reloading of index blocks. 

The ISAM file space may be (optionally) preal located with a suitable 
specification of the RSTORE parameter (available via SET, ASSIGN, 
M:DCB and M:OPEN). 

IV. PERFORMANCE 

There are several dimensions of performance which can be examined relative 
to ISAM in UTS. First the data structures in ISAM data blocks are approxi- 
mately equivalent to those in UTS consecutive files. Thus, the read/write 
CPU time for UTS consecutive files (about 1.7ms) is an upper bound on the 
CPU time required per read/write in ISAM files. 

The second major factor to consider in performance is elapsed time due . 
to l/O accesses. There are several subfactors to consider here. First, the 
No-wait l/O capability that exists in UTS will be available to allow users 
to overlap CPU and compute time. To consider other factors, a tabular list 
of differences between Keyed and ISAM files will be illustrative. 
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Feature 



Advani-age ISAM 



Advantage Keyed 



1. Variable (Large) 
Block Sizes 
vs. fixed Block 
Size 



2. Pre-Al location 
vs. D/namic 
Allocation 

3. Key in data vs. 
Keys separate from 
data 



Sequential Processing Requires 
fewer accesses to secondary 
storage due to larger block 
sizes. 



Data kept proximate thus re- 
quiring less arm movement; File 
Deleting is faster. 

Fewer I/O accesses for sequential 
processing. Less storage occupied 
if Key is naturally part of data. 



Random Processing requires 
less channel and program 
wait time due to smaller 
amount of extraneous data 

transferred. Less main memory 
required to process file. 

Wasted Secondary Space 
kept to minimum. 



Restructuring required much 
less frequently. Files may be 
created with Keys in any order. 
Easier to update with better 
representation of updates in the 
index structure. 



The above advantages in favor of ISAM will be provided by UTS ISAM. The 
number of accesses will be reduced to the same as XOS ISAM for sequential 
processing because data blocks will be variable sized and no index need be 
consulted. The arm motion will be the same for UTS ISAM as for XOS ISAM 
if pre-al location is requested. If pre-al location is not requested, groups of 
several thousand bytes will be allocated, keeping the data relatively close 
together while retaining the space efficiency of dynamic allocation. In addition, 
any UTS ISAM file can be increased in size without explicit request. Deleting 
files will be as fast in UTS ISAM as in XOS ISAM as a concise record is kept 
of all space in the file. 



V. DESIGN APPROACH 

The UTS file management system provides a convenient vehicle for 
addition of new access methods. Much the same approach would be 
used for adding ISAM as was used in adding the Random, Consecutive, 
and ANS tape access methods. This approach is represented graphically 
in Figure VI. 

Figure VI shows that user programs request system services via CAL's 
which are uniform across access methods. CAL decoding transfers 
control to the appropriate monitor routine. If the request is an open 
or close request the common catalog is consulted and the data set is prepared 
for accessing. If the request is an access request (read, write, position, etc.), 
the appropriate content manager is called to do any access-method -dependent 
processing. Each of the content managers will coll lOQ for any physical data 
transfers required. They will also call the File Space Allocator for any needs 
to allocate secondary storage. 

ISAM, an independent module, would be added as a content manager in the 
same manner as the above mentioned coiuient managers were added. A 
Content Manager Environment Simulator is available to assist in the deve- 
lopment of the above mentioned content managers. This simulator, which was 
used in development of consecutive AM and ANS AM, provides the interfaces 
depicted in the chart (namely CAL decodes interface, physical I/O interface, 
and Allocation interface). Thus, new content managers can be debugged as 
user programs from a terminal and then integrated into the system as a working 
entity. 
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The ISAM structure would be built as outlined in Appendix B. 
Block sizes would be variable with secondary storage space used 
determined by rounding block sizes to-the next multiple of 2048 
bytes. The complexity of the structure is estimated as approximately 30-40% 
greater than that of UTS consecutive files. Thus, the amount of new 
code required for the content manager is estimated at approximately 
• * 900 words. Sequential processing of ISAM files will not require any 
accesses for index blocks. 

Neither lOQ nor the Rie Space Allocator would require any modi- 
fication as they are sufficiently general to handle the requirements of 
ISAM. CAL decoding and Catalog routines would be modified to 
recognize the new access method. 

ISAM would be added to UTS as an option available to programs whose 
data are a good fit to the ISAM structure - i.e., they can avail themselves 
of the potential performance gains and can tolerate the concomitant de- 
crease in flexibility. Keyed files would be retained for their greater flexi- 
bility and function and (in some cases) superior performance 

VI. COSTS 

These cost estimates are developed by considering the size and complexity 
of the tasks involved and by comparing this development with similar ones 
which have been completed recently. 

The major development areas are: 

1) The ISAM access method itself 

2) The automatic recovery interface to assure no loss of updates 
in the event of a crash 

3) Changes to the JCL decoding in CCI and TEL to provide for new 
parameters 
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Changes are nof required fo file backup and PCL uHlit/ processors 
since organizofion information is accurately carried on file to tape and 
tape to file operations. " 

Costs are: 

Design, development, debugging, and technical documentation - 12mm 
Availability time after start of project including integration - 6 months 
with a new system version 

These costs do not include standard overhead burden. 



Similar projects completed in the last two years which, are of similar com- 
plexity and scope are: 

development elapsed 

Random files in EOO BPM 3mm 

True consecutive files in COO UTS 4mm 



ANS Tape facilities including 
both the access method and label 
validation 



24m 



m 



4 months 

7 months 

10 months 



APPENDICES 



The following iAvo appendices show: 

Functional Description of UTS Keyed files as per the 

UTS Reference Manual. 

Functional Description of XOS ISAM as per the XOS 

Reference Manual. 
The keyed file description is presented here merely to indicate the similarity 
to ISAM and to show that ISAM contains no major function or architectural 
aspect that is not presently supported in UTS. 



APPENDIX A 
UTS KEYED FILES 



APPENDIX D. FILE ORGAMIZATlOfl 
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A fiie is an organized colioctlon of information Hiat may 
only be created, modified, or deleted through the Monitor 
system. A file has one base name but may have other names 
synonymous with it". 

Information is retrieved from a file by specifying the file 
name, password, account, and the desired record within 
the file. 

The Monitor maintains a directory of accounts that have 
files which are maintained between jobs. This Is called an 
Account Directory, and contains, with each account num- 
ber, -:■" r-':''-:: oF q Jliccioiy or files ^.termed a File Direc- 
tory) for that account. A File Directory contains, with 
each file name, an addi'ess of a table containing file attri- 
butes and disc locations for that file. The table is called 
a File Information Table. To summarize, the Monitor has 
a single Account Directory, which in turn points to a File 
Directory for each account. Each File Directory, in turn, 
points to a File Information Table (FIT) for each file. 

Each file has associated v/ith it (in the FIT) information 
controlling who may access it and how It may be accessed. 
A password and a list of v/hich accounts may reader update 
the file is recorded. Protection from unauthorized dis- 
closure is attained by checking the information carried with 
the file against the information supplied by the user. 

Changes to the file are allowed or disallov/ed based on the 
user's passv/ord and account. No accidental changes can 
occur, 

Aflle may be shared among several users providing that none 
of fhem updates the file or attempts to replace the file. 

A job cannot create a file In an account other than its own. 

FIlEORGANIZATlOr^ 

KEYED FILES 

Keyed files are those In v/hich each record has an Identi- 
fying key associated v/ith it. A key consists of a byte string, 
the first byte of v/hich states the number of bytes in the 
string. The contents of each byte may be a binary number 
or a character. 

As the file is being created, a master index is also created 
with an entry for each keyed record in the file. The entry 
contains such information as the key, disc address of the 
record, size of the record, and position of the record 
within the blocking buffer. 

The records are automatically packed into blocking buffers 
with the lost portion of the lus' rcct^rd extending into an- 
other buffer as necessary. If the record is large, il is 
v/rittcn directly from the user's area instead of being 
(Xickcd into a bjffer. !\eyed files may be accessed by 
direct or sequential access. 



CONSECUTIVE FILES 

Consecutive files are files whose records are organized in a 
consecutive manner; i.e., the user is ov/are of no identi- 
fying keys associated with the records. The records may 
only be accessed sequentially. 

As with Keyed files, a master index Is created alongwlth 
the file. The master index contains information similar 
to that for keyed files. The key will be a three-byte 
'dummy' key, created by the Mionltor, but -ransparent to 
the jser. As each new record is created in a consecutive 
file, the Monitor binarily increments the last dummy key 
to obtain a nev/ dummy key. 

The records In consecutive files are blocked identically to 
keyed files. 



MULTI-LEVEL INDEX STRUCTURES 

A multi-level Index structure Is a collection of hierarchical 
levels of index blocks, where the entries in a higher level 
point to Index blocks at the next lower level and the entries 
in the lov/est level (called level 0) point to data records. 
This Is best Illustrated byan example as shown InFIgure D-1. 

Both keyed and consecutive files have level Index blocks. 
Only keyed files can have a multi-level index structure. 
The multi-level structure Is initially built during a CLOSE 
If a keyed file has more than three level Index blocks. 

In the example shown In Figure D-1, the keyed file has 

o 15,570 records and the keys at level point to these 
data records. Based on an 1 1-byte maximum key length, 
there are 40 keys in each level block and 127 keys 
in each higher-level block. 

D 390index blocks at level 0, four index blocks at level 1, 
and one ind(?x block at level 2. The next higher-level 
is built if the last level has more than three Index blocks. 

Each entry In a higher-love! index block contains the disc 
address of an index block at the next lower level, and the 
key of the first key In that block. 

The multi-level index structure can considerably speed up 
the direct access of a large keyed file, at only a small cost 
of secondary storage space. Since the keys are ordered in 
ascending sequence, at most it would take tlnree index block 
accesses to locate a data record as shov/n in the example. 
Without the higher -level index structure, It could take up 
to 390 index block accesses. 

The user has no control over the Initiol creation of the 
multi-level index structure but he can specify. when and 
If the higher-level structure should be rebuilt. This can 
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r-trecifiod by using the NEWX option on the !ASSIGN 
-^iitrol coirr.'.and or the /v'\:GPEN and /.\:DCB proccdjres. 

'■•.? space required to hold a given file can be estimated by 
..;.olying the follov/ing rules: 

^ r)ATA BLOCKS 

■< . ■ * 

. i. Each il:.':a b!oc!c contains 2048 bytes. 

., 2. Each data granule contains one data block. 

-i 

i 3. Each data bIocI< is compact, except that all records 
5 start on word boundaries. 

. .'. Each record or record segment (if a record resides in 

more than one data block) has a level index entry 
j associated v/ith it. 

i 

j LEVEL INDEX BLOCKS 

I 1. Each index block contains 1024 bytes. 

2. Each index granule contains two index blocks. 

3. Each index block is compact except that 12 bytes are 
preempted and spare space may be reserved at user 
request. 

4. Each index entry occupies KEYM plus 14 bytes. 

HIGHER-LEVEL INDEX BLOCKS 

1. Each higher-level index block contains 2043 bytes. 

2. Each higher-level index granule contains one higher- 
level index block. 

3. Each higher-level index block is compact except that 
12 bytes are reserved. 

4. Each higher-level index entry occupies KEYM plus 
five bytes. 



The following tv/o examples show the cost to build the multi- 
level index structure, i.e., disc accesses to build it and 
disc storage required to contain it, and the saving in time 
when accessing it. 



Keys/Level Index block 



(1024-12-102) 
17 



= 53 



Example 1 



Number of records 
Record size 
Key size (KEYM) 
Spare space 

Data granules 



40,000 

60 bytes 

3 bytes 

. 1 

(40^000) (60) 
2043 



(1024x .1 = 102; 14+3-17 KEYM) 

40,000 



Level Index blocks 

Level Index granules 

(758 4- 2 = 379) 
Level 1 Index blocks 

(KEYM +5=8) 
Level 1 Index granules 



53 
379 (RAD or disc) 



= 758 



758 



(2048-12)/8 



= 3 



= 3 



This file requires a tc^a! o'" 1554 granules of storage of v/hich 
three are required to store the multi-level index. It would 
cost 761 disc accesses to build the structure when the file is 
closed. V/ith the multi-level structure, each random record 
fetch requires 3-2/3 device accesses, whereas without it 
each fetch would be 254 accesses. 



Beam pie 2 



Number of data records 

Record size 
Key size (KEYM) 
Spare space 
Keys/Index block 



Keys/higher-l evel 
Index block 



= maximum for each de- 
vice (see below). 

= 1024 bytes 

15 bytes 





(1024-12) 
29 

(20 48-12) 
20 



34 



= 101 







7242 


Item 


7232 RAD 


Disk Packs 


Number of data 


6144 


24000 


records. 






Level granules. 


91 


353 


Level 1 granules. 


2 


7 


Level 2 granules. 




1 



= 1172 



the cost to build the multi-level structure in the 7242 
example is 714 device accesses. Without the multi- 
level structure a random fetch could take 707 device 
accesses in the v/orst case; with it, four accesses ore 
required. 

The reader can easily see that the cost of storing the multi- 
level index structure is trivial and the one time cost to build 
it can be insignificant for a large file v.-hich v.'ill be read or 
updated froc|uently. 
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The following refinemcnfs of fhe build and rebuild logic have 
been made fo betfer occommodofe on-line s)'sfGTi usage. 

1. If fhe higher-level structure is being builf on-line, 
only the firsf fiiree level 1 blocks will be builh, thus 
limiring fh.e cRiounf of fime taken to build the higiier- 
level structure on-line. 

2. The higher-level structure will neverbe rebuilt on-line. 

3. V/lien a record Is accessed in a file for which only a 
partial higher-level occurs, and the record is beyond 
the end of the current higher-level, then level 1 is 
extended as the search for the desired key takes place. 

RECORD BLOCKING 

The system v/ill automatically block records for keyed end 
consecutive files in 5.12-word blocks to provide more effi- 
cient use of disc space. The user has no knowledge of this 
blocking and, v/hen reading, v/ill receive the oppropriate 
record v/ithin the block and not the entire block. 

V/hen updating a keyed file, the user may rev/rite a record 
in a size larger o\- sina.Mer than the original record size. If 
necessary, the Monitor will allocate additional disc space 
to accorrmiodate the larger size. 

A write with a byte count v/ill result in a master index 
entry for the record v/ith fields in the entry pertaining to 
disc address, record size, and displacement Into the block- 
ing buffer all set to zero. 

RANDOM FILES 

Random files provide an organization for those users desiring 
to manage their ov/n files or v/ho donot v/ish to incur the over- 
head imposed by system file management. Random organiza- 
tion differs from keyed and consecutive organization osfollov/S: 

. 1. A Random file is simply a collection of contiguous 

granules on the specified device type. The number of 
granules is specified at the time the file is opened (and 
may not be expanded after it has been opened). If the 
requested number of granules are not available con- 
tiguously, an abnormal code (major code X'Ol', sub- 
code X'OB') is returned to the user and the file is not 
opened. 

2. The user must specify a relative starting granule num- 
ber with each read or write and a byte count (fhe de- 
fault byte count in the DCB may be used). If the 
starting granule number does not fall betv/een and 
the total number of gronules allocated at "OPEN" -1, 
inclusive, an error code of X'42' Is then returned to the 
user. If the byte count exceeds granule size, the oper- 
ation v/ill continue in the next coiitiguous grcnule(s) 
until all requested bytes hove been transferred. 1 he sys- 
tem v,'ill return the next available relative granule num- 
ber to fhe user (in the KBUF field of the DCB) at the 
completion of each rcad/v/rife. If there are not suffi- 
cient granules to acccmmodafe the specified byte count, 
on error code frrajo; ccd-j X'57', subcode X'4A') Is re- 
turned to the user and thic actual number of bytes trans- 
rnilledis placedin the RWS and ARS fields of the DCB. 



3. Each write/read consumes the entire specified pi- 

The contoMts of the grariule Includes no system i;,: 
tion. Management of the user's data is the lespcr-/. 
bllify of that user. 

4. Function has the following meaning for Random :i! , 
when any random file is opened it is first checked fo; 

. existence. 

o If the file does not exist and Function is IN or 
INOUT, an abnormal code of X'03' is given, i- 
the file does not exist and OUTor OUTIN Isspecl- 
fled, a nev/ Random file is allocated unless the 
associated account number differs from the uso r 



account pi.irphr>r ^'n this 



' f~ ,-, fr ' .- . 



o ".viii iioi re- 



opened and an abnormal code of X'14' v/III be- 
returned). 

o If the file does exist, the user is checked for an; • 
priafe access permission (read/v/rite account n.- - 
bers, password), and an abnormal code X' 14' ii 
returned if there Is a violation. If there Is r 
violation, the user may proceed to read (unless 
opened OUT) or write /"unless opened IN). If • . 
file is opened OUT or OUTIN, the function Is 
changed to INOUT. Note that the user may wiir* 
in a granule in v/hich he has already v/rltten, crd 
may also read a granule in which he has notwil"- 

Thus, the Monitor provides allocation of granules, security 
checks end normal I/O queuing service and clean up. ' 
user Is responsible for record management. 



FILE ACCESS 

DIRECT ACCESS 

Direct access may be used only on files with keyed 
organization. 

OUTPUT FILES 

When a WRITE is given, a key must be specified. The '<o-.^ 
do not need to be given in a sorted order. They v/ill be c- 
dered as they are stored on disc. 

Unlike sequential output files, a V/RITE never causes for- 
ward Information to be deleted. 

Reading Is not allowed. 

SCRATCH FILES 

A scratch file is Identical to an output file, except thot 
reading is permitted before fhe file is closed. As for out- 
put files, a key rnusf be specified on each Write. The 
keyed record is merged into the file. 

A Reod may or D^ay not specify a key. If a key Is 
specified, a search Is rnade of fhe file until the key ''^ 
found and the record is then read. If the key Is not 
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APPENDIX B 
XOS ISAM FILE 



the case of dlrecf-access media the actual physical disposi- 
tion of information is deyemnined solely by the systetn arid 
thus is transparent to the user. (The use of the basic direct 
access method, BDAM, implies no file structure or organi- 
zation whatever, and can only be used with private or non- 
standard disk packs.) 

The four possible file organizations, and the medio to which 
they apply, are 

o Sequential (C) — all media. 

o Indexed-Sequential (I) — direct-access only. 

o Partitioned (P) — direct-access only. 

o Direct (D) — direct-access only. 

Although, In general, each file organization corresponds to 
a particular access method for file creation, several access 
methods may apply for subsequent access to a file of given 
organization. For example, a partitioned file can be 
read by the assisted sequential, assisted partitioned, virtual 
sequential, and virtual direct access methods. 



SEQUENTIAL (C) ORGANIZATION 

The sequential file organization pemnits sequential access 
to the records or blocks of a file. It is created by either 
ASAM or VSAM, and Is the only organization applicable 
to nonmagnetic device files as v/ell as to files on magnetic 
media. 

Depending upon file media restrictions, any of the three 
record formats, F, V, or U, ore allowed with use of the 
assisted sequential access method. Although a sequential 
file can be Vv'ritten or read at the logical-record level by 
ASAM, it can be read (or .',,',:. ,a) _..!)• -'t hie block level 
by VSAM. ' 

Existing sequential files on magnetic tope are always ex- 
tendable — at the cost of losing any subsequent files on the 
same volume. On direct-access media, they may be ex- 
tended up to the limits of the possible space allocation; also 
individual records may be deleted, or modified if the record 
length Is not changed. 



INDEXED-SEQUENTIAL (1) ORGANIZATION 

The Indcxed-sequentio! file organization permits either 
direct access to individual logical records Identified by 
record key, or sequential access to records in ascending 
order of their keys, starting with a specified record. A 
record key is a dato item within the record body, pro- 
vided by tlie user, which serves to uniquely identify the 
record. The location of a record specified by key is deter- 
mined (by the system) vio an index mechanism Ihat is con- 
structed ond maintained by I he system as part of the file. 



Indexed-sequential organization is applicable only to direct- 
access media. Either F- or V-format records are allowed. 
An indexed-sequential file is created using the assisted 
indexed access method (AIAM). 

The indexed-sequential organization is shown schematically 
In Figure 6-9. The file is composed of data blocks, index 
blocks, and (possibly) overflov/ blocks. Upon creation, the 
file will consist of one or more data blocks, and at least or.e 
index block. The index may be multilevel, as illustrated 
in Figure 6-9 (1st and 2nd level index blocks). The number 
of Index blocks and number of levels thereof is a function 
of block size, number of data blocks, and record-key length 
(as described below). 

Prior to file creation, the user must request allocation of 
sufficient file space to allow for oil of the data, index, and 
overflow blocks that may eventually be needed. The method 
for calculating this space requirement is described below 
under "Space Allocation". Also prior to creation, he must 
describe to the system both the beginning byte position, 
relative to byte of the record body, and the length of the 
record key by means of the DCB parameters KYP and KYL 
respectively. 

During file creation, the user must create the record keys 
and write the logical records in ascending record-key 
order (binary collating sequence); If a record is presented 
out of ascending key order it is not accepted and an ab- 
normal condition occurs. 

For each base data block written, a record-index entry i: 
automatically created. It Is composed of the record key 
corresponding to that of the last record in the data block 
and a pointer to the beginning of that block. The record- 
Index entries are blocked as are user records, and the set of 

these blocks constitute the first-level index. 

. . .^ 

For each first-level Index block written on Index entry is 
created. It Is composed of the record key corresponding 
to that of the last record-index entry in the first-level 
index block and a pointer to the beginning of that bloc'- 
These Index entries are blocked, similarly, into the second- 
level index. 

Given enough data blocks, the above process applies re- 
cursively with third, fourth, . . . , 255fh level indices pro- 
duced. In general, ot least one (partial) index block existi 
at any level when two or more blocks exist at the next lowc' 
level — Including the "data block level". 

Overflow data blocks ore created if, during subsequent, up- 
dating, either inserted or lengthened records cause orlgino 
records to be "pushed down" beyond the boundary of a dote 
block. The resulting overflow is automatically moved to c" 
overflow block v^hich is linked betv/een the two data bloCf.! 
as shown in Figure 6-9. Two or more overflow blocks con 
be linked betv/een two data blocks in this manner. I'^o^o 
that overflow blocks do not appear explicitly in the index, 
and are undesirable from the viewpoint of access speed -■"-' 
storage spoce utilization. A utility processor, REORGh •'• 
provided to effect o reorganization of overflowed indo.^ec: 
sequential files. (See the XDS Utilities Reference Manual- 
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Figure 6-9. Indoxed-Soquential Organization 
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/A \Uo end of tho file crooHon process, the system auto- 
matically inserts o dummy rocorci having the maximum 
possible key value (X'FF-..F'). This permits subsequent 
insertion of records v/ith keys areoter than that of the lost 
record originally written, effectively allowing file exten- 
sion. The dummy lost record cannot be accessed by the 
user program, however. 

Care must be taken that the key field does not overlap the 
deletion control charoctor (bytf; 0), if the latter is speci- 
fied; a program abort on file opening will occur If KYP ~ 
(oeiuuir vuiuu u; in niu cz^z. 

If the ICY ^index copyj option ot ine (v\:Ori:iN proccJuic 
is specified, tl-.t syslerf^i v.iii au!orr,:;ii colly copy tlie index 
portion of an existing file to a temporary file in secondary 
storage. This will generally result in faster direct-access 
processing time, especially if the secondary-storage medio 
is appreciably faster than the media upon which the entire 
file resides, e.g., RAD vs. disk pack. 



PARTITIONED (P) ORGANIZATION 

The partitioned organization permits either sequential 
access to the records of a file, or direct positioning to the 
beginning of a named partition of a file for subsequent 
sequential occess to the records thereof. This organization 
is essentially an arrangement of a sequential file into 
uniquely locatable subfiles. 

A partitioned file is created v/Ith the assisted partitioned 
access method (APAM). It is applicable only to direct- 
access media. Either F or V record format may be utilized. 

The partitioned organization is shown schematically In 
Figure 6-10. Note that the user's data blocks are preceded 
by, and possibly interspersed with, system-constructed 
partition key (name) and a pointer to the first logical record 
in the associated partition. The directory blocks are, 
however, transparent to the user's program as they are 
not accessible via assisted access methods. For example, 
if either APAM or ASAK\ Is used to read a partitioned file, 
they will "skip over" the directory blocks. Hov/ever, the 
user must, prior to file creation, request allocation of 
sufficient file space to allow for both data and directory 
blocks. The method for calculating and specifying this 
space requirement is described belov/ under "Space 
Allocation". 



During subsequent processing of the file, synonym keys 
may be added, any key rr.c/ be delot-,^d, z-.d nev/ parti- 
tions created. In addition, existing records may be delet.^ 
or be modified If the record length is not changed. 

It is Important to note that when reading a partitioned file 
(either v/Ith APAM or ASAM), the system does not detect 
an end-of-partition condition: the user may read to ena- 
of-fi!e, across partition boundaries, whether starting froni 
a partition boundar/ or from beginning-of-file. A partltlc- 
key locates the beginning of o partition, but not the ere ;■ 
the preceding one: therefore a partitioned file may be 
given a hierarchical, or "nested", structure by the appro- 
priate orderino of subsumed partitions. (End-of-partitic", 
may, of course, be signaled by a user datum cerected oy 
the program, e.g., a zero-length record in \'-format. ) 

The pointer portion of a directory entry contains the rela- 
tive block number of the block in v/hich the ossoclcr;:. 
partition begins, and the byte dispiacerrjent of that part'-' 
tion's first record. (Synonym entries contain, in addition, 
a synonym indicator.) The partition keys are sorted, v/he- 
necessary, and maintained in ascending order of key va'-je 
within the directory block chain. 



DIRECT (D) ORGANIZATION 

The direct organization permits direct access to blocks of c 
file by relative block number (in relation to the beglnnirg 
of the file, block 0), via the VDAM access method only. It 
is an "unmanaged" organization relative to the C, 1, and ? 
organizations. 

A direct-organization file is composed of blocks of BKL 
defined (or default 1024-byte) length, and transmission 
must begin on a block boundary. Hov/ever, the length of 
the data actually transmitted is specified in the MiREAD cr 
MrV/RITE procedure by the transfer-length (TRL) option 
(default = 1 block). The length of data transmitted may be 
less than the block length, or may extend over several con- 
tiguous blocks, but it is limited by the maximum-transfer- 
length (MXL) parameter of the DCB associated v/ith the fre- 

No block header Is created in D organization; no loglcc- 
record structure within the block'is recognized by VDAM. 

Files created by VDAM are accessable by VSAM and also 
by ASAM using U record format. 



To create a partitioned file, the user must begin by assign- 
ing a partition key (v.'ith the M:STOV/ procedure); that is, 
the file rrsust contain at least one partition. During crea- 
tion, the user may create as many partitions os desired. In 
oddition to principal (i.e., first-cssigncd) partition keys, 
the user moy assign synonym keys, i.e., allaces of a 
given partition name. The keys may be up to 255 bytes in 
length. 



TEMPORARY AITt) PERr»lAfiErn FILES 

Files on magnetic media can be either temporary or per- 
manent files. (The distinction is not relevant for nonrrcc- 
netlc device files, for a number of reasons.) In principio. 
a permanent file is one that continues to exist in c re- 
trlcvobh." fon-n after the execution of the job that crc-ct-'J 
it; a temporary file docs not. That is, a permanent 
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for file reference, only the STS option — with OLD or 

MOD specified — need be added. 



via t ,\jL, liL.L.\j^ti i itJli 

space is a!iocatcd for the creation of a new disk or RAO 
file Qccoroing ro ina specified or default volues of the 
jIZ parameter of the lASSIGN control command (or of 
the MrASSiGN procedure, if used). The syntax of the SIZ 
parameter is described in Chapter 3. 

The meaning and effect of the SIZ poran-'oter value? vop.' 
Qccoiutng lo ine organization of the file to be created. 
They arc c!-;scribcd for each organization in the following 
subsections. 

Note thot no new space allocation can be made for a disk/ 
RAD file that is to be rewritten, i.e., a file replacing on 
existing identically-named one will occupy the same 
space olloc'jied to the original file. (Status OLD; Output 
processing mode. ) 



METHOD OF CALCULATING THE NUMBER OF INDEX 
BLOCKS REQUIRED 

Since the indexed-sequential organization is relatively corn« 
plex, a meth.od is presented for the calculation of the num- 
ber of index blocks that will be needed (assuming that the 
substantive file grows to full size), given a specified 
amount of space for base data blocks (value 1 — value2 X 
8, 192 bytes). 



Preliminary Definitions. BKL defines the block length of 
base duta blocks, overflow data blocks, and index blocks, 
in bytes. Since blocks of an indexed-sequential file, re- 
rr'-''\c" cf rcccr'J ^z:::.-^'., cciiialn g -r-uyrc: uiock header 
and a 4-byte linkage field, the usable block size, b, is 
defined as follows: 

b = BKL - 8 

KYL defines the record key length, in bytes. 

Both BKL and KYL are specified in the DCB corresponding 
to the file to be created. 



CQNSSCUr.VE ORGANIZATlOfi 

Value! of the SIZ parameter specifies, in quanta of 8K 
bytes, the initial amount of spoce to be allocated to the 
nev/ file. Value2 specifies the size of the increments to 
be added lo the file in case of either overflow of the initial 
allocation during creation, or extension of the file during 
subsequent status-MOD, Output-mode processing. 

If all of the space specified by valuel is not available on 
the first of a series of volumes specified, the remainder will 
be allocated on succeeding volume(s). 



Values to be computed; 



N , the number of base data blocks, 
o 

E , the number of index entries per index block. 
x "^ 

N,, the number of first-level index blocks. 



th 
N. , the number of i level index blocks. 



If^DEJtED-SEQUufiTlAL OaGAfJIZAHOr^ 

For an Indexed-.sequentiol file: 

valuel specifies, in quanta, the space to be 

allocated for base data blocks. 

value2 specifies in quanta, the space to be . 

allocated for (1) the creation of index blocks, and 
(2) overflow blocks. 

Note that unlike C and P organization files, the entire 
and final omount of file space — including possible over- 
flow space — is allocated at file creation time; no sub- 
sequent extensions are allowed. 

If va!uo2 is omitted or given a zero value, the system 
reserves index-block spoco as if oil of the file, were to 
be allocalcd for Lk:. j data blocks (less index space), and 
doQ$ not allow any spoce for overflow. 



Equations. Assuming that (value 1-value 2) > 0, the value 

N is derived as follows: 
o 

^ ^integer j (value l-volue 2)1 _ ^^^^ 

o portion of L BKL J 



Any index, entry is composed of a record key plus a 
3-byte pointer to a late block. Therefore, the index-entry 
length, 1, is given by 

i = KYL + 3 

and the number of entries per index block, E , by 

*^ X 

E^ = integer y 

th 
The number of n level index blocks, N,,, is given by 

the number of base dafo blocks divided by fiie number of 
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(.retries in an index block, the result being rounded 
upwards to on integer, i.e.. 



N, = integer 



N 



Similarly 



E 

L X 



N. 



N_ =" integer 



E 

I- X 



N. = integer 
« 



N 



- 1 



E 

I- X 



Therefore, the total number of index blocks, N, that will 
be created is 



To arrive at appropriate SIZ parameter values for a parti- 
tioned file the user should note that 

1. The directory entries and the data records are kept 
in separate blocks and that in both cases the effective 
block length is BKL-8. 

2. To compute the number of partition key entries a 
block can contain, Ej, one must consider that the 
length of each entry is, in bytes 



KYL + 5 



Thi 



Ej- integer [^l^^^^j 



The probable maximum number of partition keys, both 
principal and synonym, to be stowed is therefore o factor in 
estimating the best allocation. 



N = ? N. 



and the total amount of space that will be reserved there- 
fore, is in bytes 

N X BLK 

The excess of value2 x 8, 192 over the amount of space 
derived above v.'ill be available for overflov/ blocks. By 
making a preliminary esflrr.ate of value 1 and value 2, based 
on the prediction size of the data portion of the file, and 
then performing the calculations described above, the values 
of value 1 and value 2 may be adjusted to produce the 
most efficient allocation. 

Caution: Since blocks on direct-access media always 
being on a sector boundary and take up as 
many full sectors as are required to accommodate 
the. block length, the user must carefully relate 
BKL and the sector size of the device involved 
in order not to waste direct-access medio space. 
Specifically, if BKL is not equal to of a multi- 
ple of sector size, the equation given above 
for deriving N , and thus the entire calcula- 
tion, is invalidated. 



PAnTlTiOKED ORGAfllZATION 

Value 1 of the SIZ parameter specifics, in quanta, the 
initial amount of spoce 1o bo allocated to t'ne ncv.' file. 
VQlue2 specifies the size of the increments to be added 
, io the file in case of citiior overflov/ during creation, or 
extension durino subsequent siatus-MOD, Output-mode 
processing. 



DIRECT ORGANIZATION 

Value! is the total amount of space, in quanta, to be 
allocated to the direct-organization file. Value2 is not 
significant for thisorganlzation; i.e., a direct-organization 
file is not extendable beyond its creation-time allocation. 



RULE FOR ALLOCATION OF f^lULTlVOLUf/iE 
DIRECT-ACCESS FILES 

For all files necessitating parallel mounting, i.e., multi- 
volume direct-access files (I,P, or D organization), the 
amount of space specified by valuel of the SIZ parameter 
must be available exclusively on the volumes specified for 
mounting or the allocation request will be refused. 



PASSWORD PROTECTIOM 

when creating a file that is to be passv/ord protected, or 
v/hen accessing a file that has been passv/ord protected, 
an Xl-closs abnormal return occurs at OPEN time. The 
abnormal return routine must detect abnonmal code X'19' 
and must then load registers 6 and 7 v/ith a value represent- 
ing the password before executing on M:RETURN. If the 
file is being created, the password is entered into the ttDR3 
label. If an existing file is not being accessed, then the 
value is compared with the file's passv/ord in the HDR3 lobcl; 
and if the values are identical, processing continues nor- 
mally. When the passwords do not match, the job step is 
aborted. 

At file creation time, the user infoiTns the system thot a pa%s- 
v/ord is to be applied to the file by moons of the PAS option 
of the PRT (protection) field of the I ASSIGN command. 
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